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Consumer  Choice  and  Health  Care  Quality 


Consumer  choice  is  a  key  approach  to  improving  the 
quality  of  health  care  for  Americans.  The  idea  is  this: 
Survey  people  about  the  quality  of  their  health  care 
experiences  and  report  the  results  so  that  consumers  can  make 
informed  decisions  about  a  variety  of  health  issues,  such  as 
selecting  a  health  plan. 

A  project  known  as  CAHPS®  is  furthering  the  goals  of 
consumer  choice  by  developing  reliable  survey  and  reporting 
tools.  CAHPS  began  in  1995  as  a  way  to  collect  and  report 
consumer  information  about  the  quality  of  health  plans,  but 
its  objectives  have  expanded  to  include  other  areas  as  well, 
such  as  chiropractic  and  nursing  home  care.  Its  original  name — 
Consumer  Assessment  of  Health  Plans  Study — has  been 
dropped,  and  the  acronym  is  now  a  stand-alone  term.  CAHPS 
is  a  registered  trademark  of  the  Agency  for  Healthcare 
Research  and  Quality  (AHRQ),  the  project’s  sponsor. 

The  CAHPS  tools  are  continuously  assessed  and  refined, 
a  process  that  is  helping  to  inform  the  debate  about  consumer 
choice.  This  research  brief  describes  the  lessons  learned  during 
the  initial  CAHPS  effort  to  develop  survey  and  reporting  tools 
for  health  plans.  Key  insights  include  the  following: 

•  Health  plan  quality  does  count  with  consumers .  However,  con¬ 
vincing  more  Americans  to  use  information  about  quality 
will  take  time.  People  are  more  accustomed  to  making  health 
plan  choices  based  on  costs  and  benefits. 

•  Challenges  remain  in  getting  large  segments  of  the  population  to 
use  information  about  quality  when  it  is  available.  Barriers 
include  limited  education  and  reading  skills,  and  age-related 
physical  and  cognitive  declines. 

What  Is  CAHPS®? 


CAHPS  has  its  roots  in  a  study  commissioned  in  1994  by 
AHRQj:o  evaluate  existing  health  plan  satisfaction  surveys. 
The  study  found  that  most  surveys  were  not  designed  to  pro¬ 
vide  information  for  consumer  choice.1 


Armed  with  these  results,  AHRQfunded  the  initial  five- 
year,  $15  million  CAHPS  project  beginning  in  1995.  The 
project’s  goals  were  (1)  develop  a  standardized  survey  to  collect 
feedback  from  consumers  about  things  important  to  them 
in  a  health  plan  and  (2)  present  survey  results  to  consumers  in 
an  easily  understood  format  for  use  in  making  health  plan 
decisions. 

AHRQawarded  cooperative  agreements  to  three  institu¬ 
tions  to  carry  out  CAHPS: 

•  Harvard  Medical  School 

•  RAND 

•  Research  Triangle  Institute. 

In  addition,  Westat  was  selected  to  coordinate  activities 
among  the  CAHPS  investigators  and  to  operate  the  CAHPS 
Survey  Users  Network,  which  distributes  CAHPS  tools  and 
provides  technical  assistance. 

How  CAHPS®  Survey  and  Reporting  Tools  Work 

In  today’s  health  care  market,  choosing  a  health  plan  often 
means  picking  the  doctors,  hospitals,  and  other  providers  that 
can  be  used.  That  is  why  it  is  important  for  consumers  to 
consider  the  quality  of  a  health  plan  along  with  its  costs  and 
covered  services  when  choosing  a  plan  for  the  first  time  or 
switching  plans. 

Many  organizations  that  purchase  health  plans  for  con¬ 
sumers,  such  as  private  employers  and  government  agencies,  are 
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helping  the  decisionmaking  process  by  sponsoring  a  CAHPS 
survey.  The  sponsor  typically  hires  a  vendor  to  administer  the 
CAHPS  survey  to  a  sample  of  current  plan  enrollees.  The  sur¬ 
vey  results  are  then  analyzed  and  compiled  into  report  cards 
that  people  can  use  to  make  health  plan  decisions.  The  Center 
for  Medicare  &  Medicaid  Services,  for  example,  sponsors 
CAHPS  surveys,  and  the  resulting  health  plan  report  cards  are 
available  to  all  Medicare  beneficiaries. 

Other  CAHPS  sponsors  include  the  U.S.  Office  of  Per¬ 
sonnel  Management  and  the  Department  of  Defense;  state 
Medicaid  offices;  private  employers  and  business  coalitions; 
and  health  plans  (for  internal  quality  improvement  or  for 
accreditation). 

The  CAHPS  survey  and  reporting  tools  are  in  the  public 
domain  and  are  available  without  cost  through  the  CAHPS 
Survey  Users  Network  (see  “For  more  information”  on  back). 

CAHPS®  Surveys 

Unlike  many  existing  consumer  surveys  that  only  ask  people  how 
satisfied  they  are  with  their  health  plan,  a  CAHPS  survey  probes 
deeper,  posing  questions  that  are  meaningful  to  individuals  (see 
sidebar,  “CAHPS  Survey  Questions,”  on  page  5).  The  survey 
asks  people  to  rate  their  health  care  and  health  plan  overall 
(called  global  ratings)  and  to  report  on  their  health  care  experi¬ 
ences.  Results  are  summarized  in  five  different  aspects  of  care: 

•  Getting  needed  care 

•  Getting  care  quickly 

•  Doctors  who  communicate  well 

•  Courteous  and  helpful  office  staff 

•  Health  plan  customer  service. 

The  first  version  of  the  CAHPS  survey  was  released  in 
1997.  An  updated  version,  CAHPS  2.0,  was  released  a  year 
later,  incorporating  the  results  of  field-testing  and  other  analy¬ 
ses.  This  version  consists  of  the  following: 

•  Core  questionnaires  that  apply  to  all  people  with  public  or 
private  health  insurance,  including  fee-for-service  and  man¬ 
aged  care  plans.  Both  adult  and  child  versions  of  the  survey 
are  available. 

•  Supplemental  questions  that  can  be  added  to  core  survey 
items.  Supplemental  questions  address  specific  populations 
(e.g.,  Medicaid  recipients,  Medicare  beneficiaries,  people 
with  chronic  conditions,  and  children  with  special  health 
care  needs)  or  particular  health  care  services  (e.g.,  prescrip¬ 
tions,  transportation,  and  medical  equipment). 


Based  on  additional  research  and  refinements,  an  updated 
version  of  the  core  survey  questions  will  be  available  in  the 
fall  of  2002.  In  addition,  the  beta  version  of  a  questionnaire 
about  people  s  experiences  with  group  medical  practices  is 
available.  Other  CAHPS  surveys  in  various  stages  of  develop¬ 
ment  will  expand  consumer  assessments  into  such  areas  as 
dental,  chiropractic,  and  nursing  home  care,  as  well  as  behav¬ 
ioral  health  care. 


CAHPS®  Reporting  Tools 


CAHPS  provides  software  that  analyzes  answers  to  the  survey 
questions  and  summarizes  the  results  for  each  of  the  five  cate¬ 
gories  of  care.  The  software  also  generates  the  global  ratings  of 
the  health  plan  and  the  care  provided.  These  results  can  then 
be  assembled  into  a  report  card  for  consumers,  either  in  a 
printed  form  or  in  an  interactive  computer  display.  CAHPS 
provides  the  following  two  suggested  formats  for  the  report 
cards,  which  can  be  modified  for  different  audiences: 

•  Star  charts .  A  three-star  format  shows  consumers  how  a  par¬ 
ticular  health  plan  compares  against  all  others  being  offered: 
Two  stars  indicate  that  the  plans  rating  is  not  significantly 
different  from  all  others;  one  star  indicates  a  significantly 
lower  rating;  and  three  stars  indicate  a  significantly  higher 
rating.  Figure  1  shows  what  a  typical  star  chart  presentation 
might  look  like. 

•  Bar  graphs.  Individual  bar  graphs  show  the  percentage  of 
survey  responses  for  each  of  the  five  categories  of  care  and 
for  the  two  global  ratings.  Figure  2  shows  a  typical  bar  graph 
presentation  summarizing  people’s  responses  to  survey  ques¬ 
tions  about  getting  care  they  needed. 

What  Makes  CAHPS®  Different? 


Unlike  previous  consumer  survey  efforts,  CAHPS  took  a 
rigorous  approach  to  developing  a  standardized  questionnaire 
that  captures  data  from  consumers  in  a  range  of  health  plan 
options — from  fee-for-service  to  managed  care — and  in  both 
commercial  plans  and  publicly  funded  ones,  such  as  those 
offered  through  Medicare  and  Medicaid. 

In  developing  the  survey,  the  CAHPS  researchers  relied 
heavily  on  input  from  stakeholders.  Early  on,  they  held  numer¬ 
ous  focus  groups — meetings  with  consumers  and  experts — to 
find  out  what  they  thought  should  be  asked  in  the  survey.  Then, 
the  researchers  fine-tuned  the  wording  of  the  questions  by 
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Figure  2.  People's  Experiences  with  Getting  Needed  Care  in  Four  Hypothetical  Health  Plans 
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recording  how  volunteers  interpreted  them  during  one-on-one 
in-depth  interviews.  The  questionnaires  were  also  field-tested 
extensively  to  ensure  that  the  questions  were  reliable  and  valid 
indicators  of  quality  of  care  from  the  consumers  perspective. 

Using  a  rigorous  translation  process  to  achieve  high  qual¬ 
ity,  CAHPS  produced  the  questionnaires  in  foreign  languages 
commonly  spoken  in  the  United  States,  translating  them  first 
into  Spanish,  then  later  into  Vietnamese,  Korean,  Mandarin, 
and  Russian. 

Finally,  the  survey  and  reporting  tools  were  tested  under 
real-life  conditions  in  several  large  demonstrations,  including 


in  Washington  State,  Kansas,  New  Jersey,  Iowa,  Oregon,  and 
Pennsylvania. 

What  CAHPS®  Tells  Us  About  Consumer  Choice 

Today,  CAHPS  report  cards  are  available  for  health  plans 
serving  125  million  Americans.  Early  lessons  from  the  CAHPS 
project  underscore  that  using  information  about  quality  when 
making  health  plan  decisions  is  still  new  for  consumers,  who 
are  more  accustomed  to  making  choices  based  on  a  plans  costs 
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For  more  information:  The  CAHPS  Survey  Users  Network  offers  free  of  charge  a  range  of  technical  support  for  organizations  wishing  to  conduct 
a  CAHPS  survey  as  well  as  the  CAHPS  Survey  and  Reporting  Kit.  In  addition  to  all  CAHPS  survey  and  reporting  tools,  the  Kit  contains  detailed 
instructions  for  planning  and  conducting  a  CAHPS  project,  the  CAHPS  SAS  analysis  software  to  analyze  results,  a  consumer-oriented  video,  and 
print  report  templates.  The  CAHPS  Survey  Users  Network  also  provides  online  instructional  workshops,  meetings,  and  networking  opportunities.  For 
information  on  how  to  access  the  CAHPS  Survey  Users  Network,  go  to  www.cahps-sun.org. 
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and  benefits.  The  lessons  also  identify  the  challenges  ahead  in 
reaching  the  diverse  American  population  with  information 
about  quality.  Key  findings  are  highlighted  below. 

People  will  use  information  about  health  plan  quality ,  but  the 
challenge  is  getting  them  to  pay  attention  to  it  in  the first  place. 

In  laboratory  trials,  both  privately  insured  individuals  and 
those  on  Medicaid  who  read  CAHPS  report  cards  were  more 
likely  to  choose  health  plans  that  performed  better.2, 3  In  the 
real  world,  however,  consumers  are  inundated  with  informa¬ 
tion,  and  getting  their  attention  will  require  time  and  educa¬ 
tion.4  Still,  early  field  tests  of  CAHPS  report  cards  showed 
some  positive  signs.  For  example,  a  large  proportion  of  Wash¬ 
ington  State  employees  said  that  reading  the  CAHPS  report 
cards  at  open  enrollment  time  influenced  their  health  plan 
decisions  on  some  level.5  In  addition,  an  analysis  of  New  Jersey 
Medicaid  beneficiaries  who  read  the  CAHPS  report  cards  sent 
to  them  found  some  indication  that  the  information  influenced 
them  to  choose  HMOs  with  better  CAHPS  ratings.6 

The  table  below  summarizes  data  from  eight  CAHPS  evalu¬ 
ations  that  showed  the  widely  varying,  initial  responses  that  con¬ 
sumers  had  to  CAHPS  information  about  health  plan  quality. 

Limited  education  and  language  skills ,  as  well  as  age-related 
declines ,  are  barriers  to  engaging  large  segments  of  the  American 
population  in  consumer  choice. 

CAHPS  evaluations  identified  a  number  of  challenges  to 
surveying  people  about  their  health  care  experiences  and  mak¬ 
ing  the  resulting  information  about  quality  easy  to  understand 
and  use  by  others. 


•  Readability:  The  current  CAHPS  core  questionnaires  require 
a  seventh-grade  reading  level  for  full  comprehension,  exceed¬ 
ing  the  reading  ability  of  more  than  one-half  of  welfare 
recipients.9 

•  Surveying  Spanish  speakers:  Distinct  linguistic  variations 
among  U.S.  Spanish  speakers  create  significant  challenges  to 
producing  one  culturally  appropriate  survey  for  this  entire 
subgroup  of  the  U.S.  population.  An  ongoing  CAHPS  effort 
is  addressing  this  issue  by  working  with  Spanish  speakers  in 
San  Diego,  Los  Angeles,  New  York,  and  Miami.10,11 

•  Reaching  the  Medicaid  population:  The  lack  of  a  permanent 
address  for  many  Medicaid  recipients  hampers  efforts  to 
survey  them  about  their  health  care  experiences.  Mail  deliv¬ 
ery  problems  may  be  one  reason  why  only  one-half  of  the 
New  Jersey  Medicaid  beneficiaries  said  they  received  the 
CAHPS  report  during  field-testing.6, 12, 13 

•  Reaching  Medicare  beneficiaries:  39  million  Medicare  bene¬ 
ficiaries  now  have  access  to  CAHPS  ratings,  but  numerous 
issues  need  to  be  addressed  before  older  Americans  take  an 
active  role  in  choosing  their  health  plans.  The  biggest  chal¬ 
lenge  is  making  them  aware  that  information  about  quality 
exists:  Only  about  16  percent  of  Medicare  beneficiaries 
noticed  the  CAHPS  information  when  it  was  reported  for 
the  first  time  in  the  Medicare  &  You  2000  handbook  they 
received.8 

Also,  low  education  and  reading  levels,  plus  age-related 
physical  and  cognitive  declines,  make  it  difficult  for  many 
Medicare  beneficiaries  to  interpret  such  information.  After 
pilot  testing  the  new  Medicare  &  You  materials  containing 


1  Consumer  Reactions  to  CAHPS®  Report  Cards7  (in  percent) 

Health-Plan  Type 

Private  Insurance 

Medicaid 

'  . ' '  ** . . 

Medicare 

|  Looked  at  CAHPS  report 

24-77 

43 

16a 

|  15-30  min.  reading  time  (most  common) 

15-44 

38-46 

37 

|  Influenced  plan  choice  "a  lot" 

9-33 

40 

17  | 

|  Influenced  plan  choice  "a  little" 

33-39 

33-46 

21 

1  No  influence  on  plan  choice 

34-55 

15-27 

61  l 

|  aThis  percentage  is  from  reference  [8]. 
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CAHPS  information  on  quality,  the  Center  for  Medicare  6c 
Medicaid  Services  dropped  the  star  charts  altogether.  It  now 
includes  only  one  quality  measure  (doctors  who  communi¬ 
cate  well)  in  a  bar  graph  in  the  printed  booklet  that  benefi¬ 
ciaries  receive.  Interested  individuals  can  get  additional 
quality  measures  by  calling  1 -800-MEDICARE  or  using  the 
Medicare  Health  Plan  Compare  feature  on  the  Medicare 
web  site  (www.medicare.gov).7, 14,15 

The  Next  Step  for  CAHPS® 


CAHPS  will  continue  to  be  improved  and  expanded  during  a 
five-year  follow-on  study,  CAHPS  II,  which  is  scheduled  to 
begin  in  the  summer  of  2002.  Leading  the  CAHPS  II  research 
teams  are  Harvard  Medical  School,  RAND,  and  the  American 
Institutes  for  Research.  Westat  will  continue  providing  techni¬ 
cal  support.  The  goal  for  the  next  five  years  of  research  is  to 
continue  to  expand  CAHPS  beyond  health  plans  to  the  indi¬ 
vidual  provider  level,  as  well  as  to  continue  development  of 
CAHPS  instruments  for  other  health  care  settings. 

CAHPS  survey  data  are  also  being  pooled  into  a  single 
resource — called  the  National  CAHPS  Benchmarking  Data¬ 
base — for  use  by  policymakers,  researchers,  health  plans,  and 
health  plan  purchasers. 


CAHPS®  Survey  Questions 

The  following  are  typical  CAHPS  survey  ques¬ 
tions  and  possible  responses: 


Q:  On  a  scale  of  0  to  10,  how  would  you  rate 
your  personal  doctor  or  nurse? 

A:  □  0  Worst  personal  doctor  or  nurse 
possible 
□  1 
□  2 

□  3 

□  4 

□  5 

□  6 

□  7 

□  8 

□  9 

□  10  Best  personal  doctor  or  nurse 
possible 

□  I  don't  have  a  personal  doctor  or  nurse. 


Q:  In  the  last  12  months,  how  much  of  a 
problem,  if  any,  was  it  to  get  the  care  you 
or  a  doctor  believed  necessary? 

A:  □  A  big  problem 

□  A  small  problem 

□  Not  a  problem 

□  I  had  no  visits  in  the  last  12  months. 


Q:  In  the  last  12  months,  how  often  did  the 
office  staff  at  a  doctor's  office  or  clinic  treat 
you  with  courtesy  and  respect? 

A:  □  Never 

□  Sometimes 

□  Usually 

□  Always 

□  I  had  no  visits  in  the  last  12  months. 
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